Imaging based instrument event tracking

ABSTRACT

A method includes obtaining real-time imaging data of a least a sub-portion of an object and a region of interest therein. The method further includes displaying the real-time imaging data as the real-time imaging is obtained. The method further includes tracking extraction of a sample from the region of interest by an extraction device based on the real-time imaging data. The method further includes identifying an extraction location for the extracted sample based on the tracking and the real-time imaging data and generating a signal indicative thereof.

TECHNICAL FIELD

The following generally relates to imaging and more particularly toimaging based instrument event tracking, and is described withparticular application to ultrasound (US) imaging, although otherimaging modalities are contemplated herein.

BACKGROUND

An ultrasound (US) imaging system includes a transducer array thattransmits an ultrasound beam into an examination field of view. As thebeam traverses structure (e.g., of a sub-portion of an object orsubject) in the field of view, sub-portions of the beam are attenuated,scattered, and/or reflected off the structure, with some of thereflections (echoes) traversing back towards the transducer array. Thetransducer array receives echoes, which are processed to generate an USimage of the sub-portion of the object or subject.

US imaging has been used in a wide range of medical and non-medicalapplications. An example is US guided biopsy. Generally, a biopsy is aprocedure in which a small sample(s) of tissue of interest (e.g.,prostate, lung, breast, etc.) is removed for subsequent examination forabnormalities such as cancer cells. In one instance, the biopsyinstrument includes a trigger configured to actuate a spring-loadedbiopsy needle. The needle is inserted through the skin and moved to thetarget tissue. For the biopsy, the trigger is actuated, which causes aportion of needle that extracts the tissue sample to move into and outof the tissue at a relatively high speed, extracting a sample.

The movement of the needle during tissue extraction can be seen as awhite flash under real-time imaging. To record the location of thesample, in one instance the clinician manually marks start and endlocations of the biopsy sample on an image based on the trajectory ofthe needle from visual observation of the event during real-time imagingby the clinician. In another instance, the event occurrence isidentified from the sound of the actuated trigger. Unfortunately, theseapproaches are susceptible to human error and inaccuracy. Thus, they arenot well-suited for monitoring the same sample site over time and/orsubsequently taking another sample at the same location.

SUMMARY

Aspects of the application address the above matters, and others.

In one aspect, a method includes obtaining real-time imaging data of aleast a sub-portion of an object and a region of interest therein. Themethod further includes displaying the real-time imaging data as thereal-time imaging is obtained. The method further includes trackingextraction of a sample from the region of interest by an extractiondevice based on the real-time imaging data. The method further includesidentifying an extraction location for the extracted sample based on thetracking and the real-time imaging data and generating a signalindicative thereof.

In another aspect, a system includes a navigation processor and aninstrument tracking processor. The navigation processor displays imagingdata of a region of interest, as the image data is acquired. Theinstrument tracking processor tracks a path of a sample extractionsub-portion into and out of the region of interest during a sampleextraction procedure based in the imaging data. The instrument trackingprocessor generates a signal indicative of a location of the extractedsample in the region of interest based on the tracked path in theimaging data.

In another aspect, a non-transitory computer readable storage medium isencoded with computer executable instructions, which, when executed by aprocessor, causes the processor to: automatically identify and record alocation of a biopsy sample based on real-time imaging data by detectingan occurrence of an intensity value change in the real-time imaging dataand computing a characteristics of the intensity value change, whereinthe intensity value change is indicative of a biopsy extraction devicemoving in and out to extract the biopsy sample.

Those skilled in the art will recognize still other aspects of thepresent application upon reading and understanding the attacheddescription.

BRIEF DESCRIPTION OF THE DRAWINGS

The application is illustrated by way of example and not limitation inthe figures of the accompanying drawings, in which like referencesindicate similar elements and in which:

FIG. 1 schematically illustrates an example tracking system for imagingbased instrument event tracking;

FIG. 2 schematically illustrates an example instrument tracker;

FIGS. 3, 4 and 5 schematically illustrate example US imaging systems;and

FIG. 6 illustrates an example method for imaging based instrument eventtracking.

DETAILED DESCRIPTION

FIG. 1 schematically illustrates a tracking system 100. The trackingsystem 100 is configured to track a moving object in a cavity based onreal-time imaging. For explanatory purposes and sake of brevity, thiswill be described below in detail in connection with tracking a sampleextraction device and extraction of a sample therewith during a biopsyprocedure, including identifying and/or recording the sample extractionlocation.

An object 104 with a region of interest 106 is in an imaging examinationregion 108. An instrument 110 includes a biopsy device 112 with a sampleextraction sub-portion 114. The instrument further includes an actuator116 configured to actuate the biopsy device 112 to acquire a sample. Forexample, where the sample extraction sub-portion 114 includes aspring-loaded needle, the actuator 116 actuates the needle to advanceand retract to extract a sample. Other approaches are contemplatedherein.

A real-time imager 118 is configured to continuously image (acquire andgenerate images) of structure in the examination region 108 such as theregion of interest 106.

However, the real-time imager 118 can alternatively image periodically,on demand, based on a gating signal, etc. The real-time imager 118 caninclude one or more of an ultrasound (US) imager, a computed tomography(CT) imager, a magnetic resonance (MR) imager, etc. The real-time imager118 produces real-time image data. As used herein, this means the imageis generated and displayed as the raw data is acquired.

A navigator 120 is configured to produce images for guiding theprocedure. The navigator 120 receives at least the real-time image dataand, optionally, a reference image. The navigator 120 identifies theregion of interest 106 in the real-time image data and/or the referenceimage. This can be achieved through a manual (i.e., via userinteraction) and/or an automatic technique. With automatic techniques,the user can override, modify, discard, etc. an automatically identifiedregion of interest.

Where the navigator receives the real-time image data, the navigatordisplays the real-time image data as the data is acquired and received.Where the navigator receives the real-time image data and the referenceimage, the navigator 120 can overlay or superimpose the real-timeimaging data of the region of interest 106 over the region of interest106 in the reference image. For this, the reference image and thereal-time imaging data can be registered over successive image frames.

In one instance, the navigator 120 automatically identifies a cloud ofkeys (e.g., landmarks) distributed within the reference image. Thenavigator 120 then registers the reference imaging data with thereal-time imaging data. An example of such an approach is described ininternational patent application serial number PCT/US13/72154, entitled“Multi-Imaging Modality Navigation System,” and filed on Nov. 27, 2013,which is incorporated herein in its entirety by reference. Otherapproaches are contemplated herein. The combined data is displayed via adisplay 128.

The reference image, in one instance, is acquired prior to theprocedure, for example, during a planning stage, from a previouslyperformed examination, etc. The reference image can be a 2D, 3D, 4Dand/or other image. The reference image can be generated by an imagingdevice 122 such as US imager, a CT imager, an MR imager and/or otherimager such as the real-time imager 118. The reference image can beobtained from the imaging device 122, the real-time imager 118, and/or adata repository 124 (e.g., picture and archiving communication system(PACS), an electronic medical record (EMR), a radiology informationsystem (RIS), etc.), e.g., over a network 126.

An instrument tracker 130 tracks the sample extraction sub-portion 114during extraction for each sample location. In this example, when thesample extraction sub-portion 114 is at the region of interest 106, theinstrument tracker 130 employs a targeting algorithm that determineswhere and when to take the biopsy sample. In one instance, thisdetermination is followed by either automatic actuation of the actuator116 through a control signal or alerting the operator to manuallyactuate the actuator 116.

In another instance, the operator can trigger the actuator 116 based ontheir assessment that a tip of the sample extraction sub-portion 114 isat a sample extraction location of interest. In another instance, theoperator can anticipate the direction and amount of both bending of thesample extraction sub-portion 114 and tissue warping to estimate whenthe tip is at the sample extraction location of interest and trigger theactuator 116 based thereon. In any case, the movement of the sampleextraction sub-portion 114 appears as a white flash in the displayedcombined imaging data.

Generally, the white flash represents a change in pixel intensity valuesdue to the presence of the sample extraction sub-portion 114. Asdescribed in greater detail below, the instrument tracker 130 tracks thewhite flash and records the results, which identifies and records thelocation at which the sample is extracted. It is to be appreciated thatknowing the extraction location allows for detailing treatment, forexample, where the sample is indicative of tumor cells. For example,such information can be used to drive focal therapy, determine and/oradjust a dose and/or a template for brachytherapy, localize a cryo, RF,microwave, thermal, radiation and/or other ablation, etc.

The results can be conveyed to a storage device 132. For example, theresults can be stored either on the fly, as they are acquired, or placedin an information table, either local or remote, for storage at the endof the biopsy. The data can be uploaded in real-time via Ethernet, USB,WI-FI and/or other mediums. For example, in one instance, the resultsare placed in a local information table, then uploaded via Ethernet atthe end of the procedure and stored as metadata in a PACS, an EMR, aRIS, etc., along with other procedure data.

The navigator 120 and/or instrument tracker 130 can be implemented viaone or more computer processors (e.g., a central processing unit (CPU),a microprocessor, a controller, etc.) executing one or more computerexecutable instructions embedded or encoded on non-transitory computerreadable storage medium, which excludes transitory medium, but includesphysical memory. However, at least one of the computer executableinstructions can alternatively be carried by a carrier wave, signal,and/or other transitory medium.

FIG. 2 schematically illustrates a non-limiting example of theinstrument tracker 130. The illustrated example includes a frameretriever 202, which retrieves successive frames of the real-timeimaging data. The instrument tracker 130 further includes frame memory204 that stores the retrieved frames. The instrument tracker 130 furtherincludes frame evaluator 206 that evaluates the frames and detects thewhite flash corresponding to the actuate sample extraction sub-portion114.

This may include comparing intensities values in successive frames andidentifying frames in which an intensity value difference exceeds apre-determined threshold indicative of a presence of the sampleextraction sub-portion 114. The instrument tracker 130 further includesa characteristic determiner 208 that determines one or morecharacteristics of the white flash, such as the flash boundary and/orcentroid, and automatically determines the extraction location and/or atime of the extraction based thereon.

As a tip of the sample extraction sub-portion 114 moves in and out ofthe region of interest 106 the tip location is tracked throughsuccessive frames. Knowing a maximum travel distance of the sampleextraction sub-portion 114 relative to the biopsy device 112 allowscalculation of a location and a time of the actual sample extraction.This can be achieved, e.g., by extrapolation between a frame in which adistance decrease begins and a prior frame, in which a distance is stillincreasing. Other approaches are also contemplated herein. The range ofthe tracking can be limited based on an expected trajectory of thesample extraction sub-portion 114, which can be based on an image basedtracking technology, a needle guide, etc.

The location of the extracted sample can be identified (e.g., withalpha-numeric, graphical, and/or other indicia) on the reference image,one or more frames of the real-time imaging data, the combined image,etc. and/or otherwise conveyed, as described herein. Furthermore, textdescribing the location of the extraction can be incorporated into anelectronically formatted report, for example, created during and/orafter the biopsy procedure, etc.

FIG. 3 illustrates an example where the real-time imager 118 is an USimaging system, which includes a console 302 and a separate UStransducer probe 304 that interfaces therewith.

The ultrasound transducer probe 304 includes a transducer array with aplurality of transducer elements 306. The transducer array can belinear, curved, and/or otherwise shaped, fully populated, sparse and/ora combination thereof, etc. The transducer elements 306 can be operatedin 2D and/or 3D mode. The transducer elements 306 transmit ultrasoundsignals and receive echo signals.

An instrument guide 308, such as a biopsy needle guide, is affixed tothe US transducer probe 304 through a coupling 310 such as a bracket,clamp, etc. In one instance, the biopsy needle is supported in theinstrument guide 308 in a retracted position until a target tissue ofinterest is located with the US transducer probe 304 as describedherein. Then, the needle is advanced to acquire the sample of the targettissue of interest.

Transmit circuitry 312 selectively actuates or excites one or more ofthe transducer elements 306. More particularly, the transmit circuitry312 generates a set of pulses (or a pulsed signal) that are conveyed tothe transducer elements 306. The set of pulses actuates a set of thetransducer elements 306, causing the transducer elements 306 to transmitultrasound signals into an examination or scan field of view.

Receive circuitry 314 receives a set of echoes (or echo signals)generated in response to the transmitted ultrasound signals. The echoes,generally, are a result of the interaction between the emittedultrasound signals and the object (e.g., flowing blood cells, organcells, etc.) in the scan field of view. The receive circuit 314 may beconfigured for spatial compounding, filtering (e.g., FIR and/or IIR),and/or other echo processing.

A beamformer 316 processes the received echoes. In B-mode, this includesapplying time delays and weights to the echoes and summing the delayedand weighted echoes. A scan converter 318 scan converts the data fordisplay, e.g., by converting the beamformed data to the coordinatesystem of a display or display region used to visually present theresulting data.

A user interface (UI) 320 include one or more input devices (e.g., abutton, a knob, a slider, etc., touchscreen and/or physical mechanicaldevice) and/or one or more output devices (e.g., a liquid crystaldisplay, a light emitting diode, etc.), which allows for interactionwith the system 118. A display 322 visually displays the US imagingdata.

A controller 324 controls the various components of the system 118. Forexample, such control may include actuating or exciting individual orgroups of transducer elements of the probe 304 for an A-mode, B-mode,C-plane, and/or other data acquisition mode, steering and/or focusingthe transmitted signal, etc., actuating the transducer elements 306 forsteering and/or focusing the received echoes, etc.

The US probe 304 and the display 322 are physically separateelectromechanical components with respect to the console 302. The USprobe 304 and the display 322 communicate with the console 302 throughcommunications paths 326 and 328. The communications paths 326 and 328can be wired (e.g., a physical cable and connectors) and/or wireless.

FIG. 4 illustrates a variation of the real-time imager 118. In thisexample, the console 302 includes a single housing 402. The singlehousing 402 houses and physically supports the transducer elements 306,the instrument guide 308, the transmit circuitry 312, the receivecircuitry 314, the beamformer 316, the scan converter 318 and thecontroller 324, all of which are inside the single housing 402.

The user interface 320 and/or the display 322 are part of the housing402. For example, the display 322, in one instance, is a sub-portion ofone of the sides of the housing 402. The user interface 320 may includephysical mechanical controls at other locations of the housing 402. Anultrasound window 404 is also part of or integrated with the console302. In this instance, the transducer elements 306 are disposed in thehousing 402 behind the ultrasound window 404 and emit signals andreceive echoes there through.

In FIG. 4, the real-time imager 118 is a hand-held ultrasound apparatus,which uses internally located power, e.g., from a power source such as abattery, a capacitor, etc. to power the components therein, and/or powerfrom an external power source. An example of a hand-held device aredescribed in U.S. Pat. No. 7,699,776 to Walker et al., entitled“Intuitive Ultrasonic Imaging System and Related Method Thereof,” andfiled on Mar. 6, 2003, which is incorporated herein in its entirety byreference.

An example of hand-held ultrasound apparatus with an internal instrumentguide is described in Ser. No. 13/017,344 to O'Connor, entitled“Ultrasound imaging apparatus,” and filed on Jan. 31, 2011, and anexample with an external instrument guide is described in U.S. Pat. No.8,226,562 to Pelissier, entitled “Hand-Held Ultrasound System HavingSterile Enclosure,” and filed on Aug. 7, 2008, both of which areincorporated herein in their entirety by reference.

FIG. 5 illustrates a variation of FIG. 4 in which the instrument guide308 is disposed out of the single housing 402 and affixed theretothrough the coupling 310.

Although the approach described herein does not require the use of anexternal tracking system and/or electro-mechanical sensors, an externaltracking system and/or electro-mechanical sensors can be used with theapproach described herein. For example, where the instrument guide isnot part of the real-time imager 118 or affixed thereto,electro-mechanical sensors affixed to the instrument can be registeredto the 3D non-US imaging data coordinate system and used to the trackthe instrument.

FIG. 6 illustrates an example method for imaging based instrumenttracking.

It is to be appreciated that the ordering of the following acts is forexplanatory purposes and is not limiting. As such, one or more of theacts can be performed in a different order, including, but not limitedto, concurrently. Furthermore, one or more of the acts may be omittedand/or one or more other acts may be added.

At 602, a reference image of an object and a region of interest thereinare obtained.

At 604, real-time imaging data of the object and the region of interestare obtained.

At 606, the reference image and the real-time imaging data of the regionof interest are combined and displayed.

At 608, extraction of a sample from the region of interest by anextraction device is tracked based on the combined data, as describedherein.

At 610, the extraction location is identified based on the tracking andthe real-time imaging data.

At 612, the extraction time is identified based on the tracking and thereal-time imaging data.

The extracted location is stored, processed, and/or otherwise utilized,for example, to tailor treatment of a tumor.

In a variation, act 602 is omitted, and the tracking is based on thereal-time imaging and not the combined data or reference image.

The above may be implemented by way of computer readable instructions,encoded or embedded on computer readable storage medium, which, whenexecuted by a computer processor(s), cause the processor(s) to carry outthe described acts. Additionally or alternatively, at least one of thecomputer readable instructions is carried by a signal, carrier wave orother transitory medium.

The application has been described with reference to variousembodiments. Modifications and alterations will occur to others uponreading the application. It is intended that the invention be construedas including all such modifications and alterations, including insofaras they come within the scope of the appended claims and the equivalentsthereof.

What is claimed is:
 1. A method, comprising: obtaining real-time imagingdata of a least a sub-portion of an object and a region of interesttherein; displaying the real-time imaging data as the real-time imagingdata is obtained; tracking extraction of a sample from the region ofinterest by an extraction device based on the real-time imaging data;and identifying an extraction location for the extracted sample based onthe tracking and the real-time imaging data and generating a signalindicative thereof.
 2. The method of claim 1, further comprising: priorto the extraction, storing frames of the real-time imaging data; duringthe extraction, comparing intensity values of successive frames of thestored frames; identifying frames in which a difference in intensityvalue satisfies a pre-determined threshold value that indicates presenceof the extraction device; and identifying the extraction location basedon the identified frames.
 3. The method of claim 2, further comprising:identifying a first frame in which a tip of the extraction device movesin the region of interest; and identifying a second frame in which thetip of the extraction device moves out of the region of interest,wherein the first and second frames are adjacent frames.
 4. The methodof claim 3, further comprising: determining a maximum travel distance ofthe extraction device by extrapolation between the first and secondframes.
 5. The method of claim 4, further comprising: computing theextraction location based on the maximum travel distance.
 6. The methodof claim 4, further comprising: computing a time of the extraction basedon a first time of the first frame and a second time of the secondframe.
 7. The method of claim 3, further comprising: determining anextraction boundary based on the first and second frames.
 8. The methodof claim 3, further comprising: determining an extraction centroid basedon the first and second frames.
 9. The method of claim 1, furthercombining comprises: obtaining a reference image of the object with theregion of interest therein; combining the reference image and thereal-time imaging data, constructing combined data; displaying thecombined data; tracking extraction of the sample from the region ofinterest by the extraction device based on the combined data; andidentifying the extraction location for the extracted sample based onthe tracking and the real-time imaging data and generating the signalindicative thereof.
 10. The method of claim 9, wherein the combiningcomprises: identifying the region of interest in the reference image;registering the real-time imaging data with the region of interest inthe reference image over successive frames; and overlaying the real-timeimaging data over the reference image based on the registration.
 11. Themethod of claim 10, wherein the registering comprises: identifying acloud of landmarks distributed within the reference image; andregistering the reference imaging data with the real-time imaging databased on the identified cloud of landmarks.
 12. The method of claim 9,further comprising: acquiring the reference image with at least one ofan ultrasound scanner, a computed tomography scanner, or a magneticresonance scanner.
 13. The method of claim 1, further comprising:acquiring the real-time imaging data with an ultrasound scanner.
 14. Asystem, comprising: a navigation processor that displays imaging data ofa region of interest, as the image data is acquired; and an instrumenttracking processor that tracks a path of a sample extraction sub-portioninto and out of the region of interest during a sample extractionprocedure based on the imaging data, wherein the instrument trackingprocessor generates a signal indicative of a location of the extractedsample in the region of interest based on the tracked path in theimaging data.
 15. The system of claim 14, wherein the instrumenttracking processor, during the extraction, identifies a frame of theimaging data in which a tip of the sample extraction sub-portionadvances in the region of interest and a subsequent frame of the imagingdata in which the tip of the sample extraction sub-portion retracts inthe region of interest.
 16. The system of claim 15, wherein theinstrument tracking processor computes an extraction location in theregion of interest based on the frame and the subsequent frame.
 17. Thesystem of claim 15, wherein the instrument tracking processor computesan extraction time based on the frame and the subsequent frame.
 18. Thesystem of claim 15, wherein the instrument tracking processor computesan extraction boundary in the region of interest based on the frame andthe subsequent frame.
 19. The system of claim 15, wherein the instrumenttracking processor computes an extraction centroid in the region ofinterest based on the frame and the subsequent frame.
 20. Anon-transitory computer readable storage medium encoded with computerexecutable instructions, which, when executed by a processor, causes theprocessor to: automatically identify and record a location of a biopsysample based on real-time imaging data by detecting an occurrence of anintensity value change in the real-time imaging data and computing acharacteristics of the intensity value change, wherein the intensityvalue change is indicative of a biopsy extraction device moving in andout to extract the biopsy sample.